1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
|
PCI broken in qemu ppc e500 in v2.12.0 and other versions
The same qemu -M mpc... command that works on qemu-system-ppc version 2.8.0 freezes guest on bootup and shows error for qemu-system-ppc version 4.2.0release and 4.19dirtygit:
qemu-system-ppc: virtio-blk failed to set guest notifier (-24), ensure -accel kvm is set.
qemu-system-ppc: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
ends/freezes at:
nbd: registered device at major 43
vda:
I'm using -drive file=/home/me/rawimage.dd,if=virtio and works fine in version 2.8.0 installed with apt-get install (Ubuntu 17.04) and also with 2.8.0 official release from git/github that I compiled/built myself. But both of the newer releases fail on the same exact machine same config.
I also noticed that qemu-2.8.0 was fine with mtd but the newer ones I tried weren't, ie gave
qemu-system-ppc: -drive if=mtd: machine type does not support if=mtd,bus=0,unit=0
(but I removed -drive if=mtd since wasn't using it anyway)
I also tried on windows but I think virtio doesn't work on windows hosts at all? On windows host it fails the same way, even version 2.12 as well as 4.1.10...
used:
./configure --prefix=/opt/... --enable-fdt --enable-kvm --enable-debug
(basically all steps the same on same exact system same config, yet 2.8.0 works fine whether apt-get installed or built from source while the others I built, 4.19/4.2.0 or 2.12/4.1.10(win) don't.)
In case newer qemu versions act weird on various kernels, I did try with both vmlinuz-4.10.0-19-generic and vmlinuz-4.13.12-041312-generic (I didn't compile them but I can provide config-..files)
tx
ecs
Also tested on another system (Debian GNU/Linux 9 \n \l with kernel SMP Debian 3.16.56-1+deb8u1 (2018-05-08) x86_64) besides the previous Ubuntu 17.04 and confirmed even Qemu 2.8.1 is working but Qemu 3.1.10 and higher not working, virtio fails/freezes guest at vda as on the other system.
Could you provide the full qemu command line?
Did you try with just a basic virtio disk and it works for you?
Because even a basic virtio drive addition fails for me, even this fails on higher than 2.8.1
For example:
qemu-system-ppc -M mpc8544ds -nographic -kernel /home/me/boot/uImage-2.6.32 -append "root=/dev/vda rw" -drive file=/home/me/mmcblk0p2.dd,if=virtio
The only thing I can think of, is if somehow vda fails due to me running the higher version qemu binaries from their build locations/paths without actually "installing" them in usual paths etc.
But I ran the 2.8 version from build location I compiled it from and it worked from there, but perhaps the 2.8 version was also the distro installed default one so maybe it found dependencies it needed?
Anyway I just now reconfigured 4.2.0 with --prefix /opt/qemu4.2.0 and ran it from installed dir:
root@myserver:/opt/qemu4.2.0/bin# ./qemu-system-ppc -M mpc8544ds -nographic -kernel /home/me/boot/uImage-2.6.32 -append "root=/dev/vda rw" -drive file=/home/me/mmcblk0p2.dd,if=virtio
But it still fails even after make install and running it from the /opt/qemu4.2.0/bin directory.
Is it somehow conflicting with the other qemu version 2.8.. installed by usual apt-get install?
Regardless of how I start them, version 3.1.0 and 4.2.0rc4 and some other 4.19git and 4.2.0final all fail/freeze at:
"
....
nbd: registered device at major 43
vda:
"
Perhaps you can try to disable the "modern" mode of virtio (The endianness of the API has been changed):
replace
-drive file=/home/me/mmcblk0p2.dd,if=virtio
by
-device virtio-blk-pci,drive=drive0,disable-modern=true \
-drive file=mmcblk0p2.dd,if=none,id=drive0,format=raw
Thanks I tried with:
/root/QEMU/qemu-git-4.2.0rc4/qemu/build/ppc-softmmu/qemu-system-ppc -M mpc8544ds -nographic -kernel /home/me/boot/uImage-2.6.32 -append "root=/dev/vda rw" -device virtio-blk-pci,drive=drive0,disable-modern=true -drive file=/home/me/mmcblk0p2.dd,if=none,id=drive0,format=raw
And again it worked with qemu 2.8.1 but failed with the above 4.2.0rc4 on the same x86_64 host.
On another x86_64 host I confirmed that the below works with qemu 2.8.0
root@myserver:~# qemu-system-ppc -M mpc8544ds -nographic -kernel /home/me/boot/uImage-2.6.32 -append "root=/dev/vda rw" -device virtio-blk-pci,drive=drive0,disable-modern=true -drive file=/home/me/mmcblk0p2.dd,if=none,id=drive0,format=raw
But again even on this system 4.2.0 failes with that same command:
root@myserver:~# /root/QEMU/qemu-4.2.0/build/ppc-softmmu/qemu-system-ppc -M mpc8544ds -nographic -kernel /home/me/boot/uImage-2.6.32 -append "root=/dev/vda rw" -device virtio-blk-pci,drive=drive0,disable-modern=true -drive file=/home/me/mmcblk0p2.dd,if=none,id=drive0,format=raw
Fails/freezes at the same vda: location.
Running it from its installed location didn't help, the following still failed at vda: also.
root@myserver:/opt/qemu4.2.0/bin# ./qemu-system-ppc -M mpc8544ds -nographic -kernel /home/me/boot/uImage-2.6.32 -append "root=/dev/vda rw" -device virtio-blk-pci,drive=drive0,disable-modern=true -drive file=/home/me/mmcblk0p2.dd,if=none,id=drive0,format=raw
Although I didn't think its required for the softmmu qemu "emulation" only, ie not "kvm", I even enabled kvm as well as DMAR+IOMMU on the kernel and recompiled 4.2.0 but had same vda: failure.
fyi from what I recall guest kernel was built using mpc85xx_defconfig with some additions like virtio etc. If virtio is working for you just fine using same command as mine, then perhaps its some peculiarity to do with my specific guest kernel or kernel version? (uImage is about 3.4M with equivalent vmlinux about 72M)
Hope you enjoyed the Holidays, Happy 2020! I would really appreciate if you could confirm for me if virtio works fine for you with ppc -M mpc8544ds with older Linux guest kernels like 2.6.32
thanks!
Could you provide your binary uImage-2.6.32?
With some precautionary measures I think I can provide it. Not sure what of our drivers may already be compiled in etc so I need to send it to you privately so only you have access for testing etc after which you would delete it once issue fixed or discovered etc. Is it possible to send you private message on here with such a link or better email? thanks
Sorry for the delay, I have sent you a private message/email with the actual kernel image. thx!
This is broken by:
commit 67113c03423a23e60915574275aed7d60e9f85e1
Author: Michael Davidsaver <email address hidden>
Date: Sun Nov 26 15:59:05 2017 -0600
e500: fix pci host bridge class/type
Correct some confusion wrt. the PCI facing
side of the PCI host bridge (not PCIe root complex).
The ref. manual for the mpc8533 (as well as
mpc8540 and mpc8540) give the class code as
PCI_CLASS_PROCESSOR_POWERPC.
While the PCI_HEADER_TYPE field is oddly omitted,
the tables in the "PCI Configuration Header"
section shows a type 0 layout using all 6 BAR
registers (as 2x 32, and 2x 64 bit regions)
So 997505065dc92e533debf5cb23012ba4e673d387
seems to be in error. Although there was
perhaps some confusion as the mpc8533
has a separate PCIe root complex.
With PCIe, a root complex has PCI_HEADER_TYPE=1.
Neither the PCI host bridge, nor the PCIe
root complex advertise class PCI_CLASS_BRIDGE_PCI.
This was confusing Linux guests, which try
to interpret the host bridge as a pci-pci
bridge, but get confused and re-enumerate
the bus when the primary/secondary/subordinate
bus registers don't have valid values.
Signed-off-by: Michael Davidsaver <email address hidden>
Signed-off-by: David Gibson <email address hidden>
If I revert 67113c03423a on top of master, vda is correctly detected.
Thanks for all the help Laurent! I'm new to git so not surre how to 'properly' revert a previous commit on top of master, so I'll google, but if you have some a good link please do send.
Also, I've heard of the term "bisect" for figuring out at which commit something breaks and if there were some good documentation to spell out the steps to do that for users that aren't, well advanced kernel gurus :D , I'm sure we'd be happy to save you smarter guys time with any mundane testing steps when possible :) thx!
Not even reverting the patch worked for me, and it's still broken on qemu 5.1.
For example:
~/OSS/qemu/ppc-softmmu/qemu-system-ppc -machine mpc8544ds -nographic -cpu e500mc -serial mon:stdio -kernel zImage -initrd rootfs.ird -append 'console=ttyS0,115200' -device e1000,netdev=main -netdev hubport,hubid=0,id=main -net tap,ifname=tap0 -device virtio-balloon-pci -device virtio-rng-pci -device virtio-blk-pci-transitional,drive=drive0 -drive file=disk,if=none,id=drive0,format=raw
causes the linux kernel to freeze after probing the virtio_blk device:
virtio_rng: probe of virtio1 failed with error -22
virtio_blk virtio2: [vda] 131072 512-byte logical blocks (67.1 MB/64.0 MiB)
Not specifying the virtio-blk-pci device makes the system boot, but still all but the first (e1000) PCI devices seem to not probe.
It seems I can trace this behavior at least to version 2.4.1, probably even sooner (can't make my linux boot on those, so I'm unsure...).
After some research, the problem is that mpc8544ds has only 2 PCI slots defined (hw/ppc/mpc8544ds.c -> pmc->pci_nr_slots = 2;). This in turn results in DTB only contain 2 devices in pci@e0008000/interrupt-map. Too bad qemu doesn't complain when more devices are added - the PCI bars seem to be OK, just interrupts are not found by linux, hence the error -22:
pci 8000:00:13.0: of_irq_parse_pci: failed with rc=-22
...and later virtio_rng probe freeze (which freezes linux boot, if a module is not used and probed in different process).
Changing pci_nr_slots to bigger number (e.g. 4) seems to work just OK, though of course the mpc8544ds simulation is then non-realistic. A cleaner solution is adding PCI-PCI bridge, that seems to work too.
As a side-note, MSI doesn't seem to work on e500mc neither. Enabling MSI support in kernel seems to cause that virtio-blk-pci device probe freeze in linux, /proc/interrupts shows:
19: 0 fsl-msi-224 0 Edge virtio1-config
20: 0 fsl-msi-224 1 Edge virtio1-req.0
Without MSI, legacy IRQ is used and that seems to work OK:
17: 743 OpenPIC 3 Level virtio1
Alternatively, passing vectors=0 to the virtio device (-device virtio-blk,drive=drive0,vectors=0 -drive ...) does the trick as well.
That was a fun ride... :-)
Sorry, above I meant "virtio-blk freeze" (no virtio_rng). But in any case it's obviously not directly related to this bug, so disregard it... Sorry for the noise.
The QEMU project is currently considering to move its bug tracking to
another system. For this we need to know which bugs are still valid
and which could be closed already. Thus we are setting older bugs to
"Incomplete" now.
If you still think this bug report here is valid, then please switch
the state back to "New" within the next 60 days, otherwise this report
will be marked as "Expired". Or please mark it as "Fix Released" if
the problem has been solved with a newer version of QEMU already.
Thank you and sorry for the inconvenience.
[Expired for QEMU because there has been no activity for 60 days.]
|